RrJrraitf  RrvwJi  and  Rrutlh . .  .Wrnirtg,  fodtiij,  and  Tomorrow 


Relevant  Research  and  Results . .  .Yesterday,  Today,  and  Ibmorrow 


Intelligence,  Reconnaissance  and  Surveillance  Science  and 

Technology 

Expeditionary  Warfare  and  Combating  Terrorism 

Large  Tactical  Sensor  Networks 

Program 

Martin  Kruger,  ONR 
krugerm@onr.navy.mil 
Suzanne  Logan,  MITRE 
slogan@mitre.org 


Report  Documentation  Page 


Form  Approved 
OMB  No.  0704-0188 


Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
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Program  Objective 

Enable  Tactical  Persistent  ISR,  Relevant  to  the  War  on 
Terror 

•  Sensor  Models 

-  T ranslate  threats  to  detectable  features 

-  Translate  a  specific  sensor  to  its  ability  to  detect  relevant  features 

•  Smart  Sensors 

-  Processing  at  the  sensor  node  to  translate  raw  data  to  useful 
information 

-  Indexed  distributed  data  structure  that  enables  sensors  nodes  to 
have  “context” 

•  Service  Oriented  Sensor  Network 

-  Hardware  and  software  that  enable  current  or  planned  sensors  to 
transition  to  a  netcentric  environment 

-  Enable  distributed  control  of  persistent  ISR  assets 

Relevant  raw  data  to  useful  information  for  tactical 

situational  understanding 
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Fusion 

-  Aggregation  algorithms,  rules  &  formulas,  pattern  definition  and 
analysis,  space/time  correlation  to  events 

-  Match  analytic  resources  to  volume  of  persistent  ISR  assets 

Automated  Tactical  Platform  &  Sensor  Planning  and 
Management 

-  Algorithms  to  control  and  direct  persistent  ISR  assets 

-  Aligned  set  of  multi-INT  collection  plans 

-  Software  enabling  dynamic  sensor  management 

Human  to  Sensor  Field  Interface 

-  Alert  quick  reaction  forces 

-  Interface  between  the  operator  and  knowledge  repository 

Local  Tactical  Net  &  DCGS  Integration 

-  Smart  agents  that  manage  bi-directional  flow  of  data,  useful 
information  and  knowledge  between  DCGS  and  the  tactical  user 


Relevant  raw  data  to  useful  information  for  tactical 

situational  understanding 
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Semantic  Web 


-  Create  an  ontology  which 
provides  a  standard 
representation  that  users 
can  query  for  meaningful 
information. 

-  Create  a  set  of  standards, 
tools  and  test  bed  that  will 
demonstrate  how  a  tactical 
ontology  and  semantic- 
based  sensor  networking 
can  dramatically  improve  the 
effectiveness  of  intelligence 
resources 
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Statistically  Determine  TTPs 


*  Create  Statistical-based  modeling  of  features  to  sensor 

modalities 

*  Choice  of  Features  for  Reduction 

-  Determination  of  the  importance  of  each  feature  required 

-  Use  metrics  that  compare  the  effect  that  reduced  subsets  of 
features  have  versus  using  the  larger  datasets 

*  Placement  of  Sensors 

-  Model  the  capability  of  each  defined  sensor  to  acquire 
detectable  features  and  characteristics 

-  Optimal  placement  of  sensors,  even  with  full  information  on 
existing  and  expected  threat 

-  Forecast  and  visibility  models  for  each  sensor 

-  Placement  algorithm  for  sensors.  Sensor  type  database  with 
range  profiles,  sensitivity,  uncertainty  will  be  utilized 


Statistically  Determine  TTPs 


•  Innovative  geospatial  analysis  and  asymmetric- 
threat  modeling  techniques  for  urban 
environments 

-  Foundation  of  technique  is  associations  made 
between  event  data,  sensor  data,  and  contextual 
information  sources 

-  Technique  excels  at  indicating: 

•  Greatest  level  of  threat  for  GWOT  mission 

•  Level  of  threat  expected  at  each  point  along  a  travel  path 

•  Greatest  likelihood  collection  areas  for  observing  a  target 

-  Technique  reduces  the  search  area  required  and 
maximizes  the  placement  of  resources 
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Statistically  Determine  TTPs 

Geospatial  Predictive  Analysis 
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Case  1 :  Threat  Prediction 
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Entity  Tracking 


-  Distributed  Smart  Sensor  EO  nodes  for  Wide  Area 
Surveillance 

-  Vehicle  signature  based  tracking 

-  Matching  people  to  HUMINT  descriptions 

-  Integrated  system  for  vehicle  tracking  and  forensic 
analysis  combining  multiple  sensors  (GMTI,  EO/IR, 
IMINTand  HUMINT) 


Entity  Tracking 


•  Distributed  air-ground  collaborative  vehicle  &  people  association  and 
data  mining  system: 

—  Provide  persistent  situational  awareness  of  an  extended  area. 

—  Provides  rapid  and  reliable  sensing  and  response  in  the  tactical 
environment. 

•  Key  Innovations 

—  Smart  Sensor  Processor  for  Moving  &  Stationary  Cameras 

—  Cross-Sensor  Cueing  and  Vehicle  Association  across  disparate  pose 

—  Multi-Sensor  Multi-Hypothesis  Track  Fusion  and  Large  Area  Data 
Association 

—  Classification,  Querying,  Mining  &  Database  Functions 
—  Distributed  Scalable  Processing  Architecture  and  System 
—  Large-area  Common  Operating  Picture 
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Send  Video  Clips 

On  request  send  n-second 
video  clip  of  any  vehicle. 


Send  Reports 
For  each  vehicle  send: 
local  ID,  time,  location,  speed, 
feature  set. 


Store  Video 
Short  sequences 
for  each  vehicle. 


Extract  Features 

Size,  color,  silhouettes, 
vehicle  parts  (side,  top,  ...), 
measurements  (wheelbase,  ...) 


Database  Query/Mining 

Build/update  database,  retrieve 
vehicles  of  specific  attributes 


Entity  Tracking 


•  MHF:  Multiple  Hypothesis  Fusion  with  map 
and  road  network  information 

-  Manage  and  combine  multiple  local  target  tracks 
to  form  multiple  globally  consistent  tracks  with 
map  &  road  network. 

-  Combine  multi-camera  based  video  tracks  with 
GMTI 

-  Essential  for  long  range  and  wide  area  automatic 
multiple  target  tracking. 
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Images  to  Network  Understanding 


Dynamic  face  recognition  watch  list 

•  Best  shot  Facial  Recognition  from  video 

Detect  suspicious  aggregate  population  or 
individual  activity  and  provide  warnings 

Detect  human  figures  from  surveillance 
video  and  perform  human  activity  statistical 
analysis  to  detect  unusual  crowd  activity 

SNA  for  terrorist/insurgent  networks,  identify 
key  individuals  in  the  organization,  and 
analyze  the  organizational  dynamics  based 
on  Video-INT 


SOA  /  Local  Tactical  Network  and 

DCGS-MC 


•  Service  Oriented  Architecture 

-  Service  Oriented  Architecture  (SOA)  for  LTSN,  based 
on  the  DOGS  Integration  Backbone  (DIB) 

-  Disadvantaged  User  Interface 

-  Create  Software  developers  toolkit  for  interfacing  to 
the  SOA  software  framework 

-  Sensor  visibility 

•  Link  to  DCGS-MC 

-  Adapt  to  small  tactical  unit  environments  with 
metadata  catalogs  at  three  simulated  PoR  sensor 
interfaces 
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COMINT  to  Network  Understanding 


•  Detect  aggregates  from  COMINT  datasets 

•  Analyze  and  propagate  suspicion  at  the  entity 
and  aggregate  level 

•  Analyze  aggregate  evolution  over  time 

•  Develop  pattern  analysis  and  event  correlation 
tool 
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COMINT  to  Network  Understanding 

•  Detect  Patterns  in  the  Data 

-  Find  threatening  activity  hidden  in  the  benign 

•  Address  Information  Overload 

-  Almost  all  activity  is  benign 

-  Faint  signal  in  lots  of  noise 

•  Operate  when  ground  truth  is  unavailable 

-  Incomplete  and  inaccurate  evidence 

-  Lack  of  observability 

•  Cannot  assume  A  Priori  Knowledge 
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Warfighter  Alerts 


•  Multi-INT  Fusion  for  Detection  of  Enemy  Activities  and  Threat  Networks 
(Probabilistic  Approach) 

-  Sensors:  Radar  GMTI,  EO/IR  Video,  SIGINT,  Wide  Area  EO,  SAR,  HSI  Imagery, 
Lidar  Scans,  HUMINT 

•  Geospatial-temporal  Pattern  Analysis  software  system  to  detect  and  alert 
threat  activities 

-  Exploit  multi-INT,  kinematic,  geospatial  and  transactional  data 

•  Server-side  network  analysis  technology  to: 

-  Fuse  acquired  network  information  with  prior  knowledge 

-  Characterize  networks  by  type  and  properties 

-  Assess  entities  and  networks 

-  Recommend  courses  of  action  to  collect  clarifying 

-  Recommend  courses  of  action  to  disrupt  enemy  networks 


Mission  Aware  Sensor  Field 


•  Real-time,  context  dependant  situational 
awareness  for  the  warfighter 

•  Mission-based  l&W  to  warfighter 

-  Investigate  methodology  for  delivery 

•  Vibro-tactile,  speech,  etc 

-  Create  methods  for  push  of  l&W 

•  CLENS  as  possible  demo  sensor 


Mission  Aware  Sensor  Field 
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Exfiltration  to  long 
haul  comms 


Image  captured  from 
digital  camera 


Target  detection, 
localization  and  track 


Target  image  fused 
with  track  information 


Exfiltration  to 
warfighter 


Directional  vibrotactile 
situational  awareness 


System  Concept:  Convert  intel  data  collection  tools  ^realtime  tactical  asse 


RrJrraitr  Rrvo/rJt  and  Rruth* . .  ,)ntrrdag,  1 bdaj,  and  Tomorrow 

Enabling  Sensor  Swarms 

•  Optimization  algorithms  and  collaborative  control 
protocols  for  allocation  of  tasks  to  UAVs 

•  Various  Airborne  platforms  with  various  sensor 
modalities:  lidar,  near  infra-red,  or  thermal  infra¬ 
red 

•  Geographically  Distributed  Users  with  different 
priority  rights  on  different  platforms 

•  Allow  for  sensor  fusion  for  multiple  intelligence 
sources  (Multi-INT) 


Enabling  Sensor  Swarms 


•  User  defines  the  mission 

•  UAV’s  define  tasks  and 

sub-tasks 

•  Philosophy 

“The  user  specifies  what 
he  or  she  would  like 
accomplished. 

The  system  decides  how 
to  do  so  efficiently.  ” 

•Previous  work  has  successfully 
flown  four  vehicles  simultaneously. 


Task  Creation 


Step  1 :  Select  task 
Step  2:  Parameters 


Priority  How  _▼] 

r 


C  Once  C  Continuous  des.  search 

period  (min) 
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Distributed  Knowledge  and 
Knowledge  Needs 


•  Tactical  Intelligence  Ontology  Development 

-  Ontology  for  All  levels  of  Command 

-  Who  needs  what  data  based  on  what  is  available  and  when 

•  Smart  multicast  support  for  dissemination  to  maximize  bandwidth  usage 

•  Smart  subscription 

•  Proof  of  Concept  Tactical  Intelligence  Community  Of  Interest 


Company  HQ 
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Interface  to  Warfighter 

•  HUMINT  via  Spoken  Interface,  Alerting  Agents 

•  Structuring  of  HUMINT  Reports 

•  Develop  System  Architecture,  Interaction 
Paradigms,  User  Interface  Mock-ups  and  specs 
for  Tactical  Users 

•  Cognitive  Task  Analysis,  Workflows 


